feat(sdk/go/ai): support the Infron gateway - #874
Conversation
|
|
meridah7
changed the title
Infron is an OpenAI-compatible inference gateway that serves the standard <provider>/<model> ids, so a model moves across by prefix alone: infron/moonshotai/kimi-k2.6 routes the same model the bare id names. Follows the provider shape already in this package rather than inventing a new one: - infron_attribution.go mirrors the existing attribution helper. Infron accepts the same HTTP-Referer / X-Title pair, and the attribution env vars already configured for the existing gateway are honored as fallbacks, so a deployment that already declares itself as 'AgentField AI' keeps that identity after switching gateways. - Config gains IsInfron(); DefaultConfig() reads INFRON_API_KEY and points at https://llm.onerouter.pro/v1. - client.go attaches attribution on both the sync and streaming paths. - marshalRequest opts Infron into native usage accounting and strips the routing-only 'infron/' model prefix before the request goes out (stripInfronPrefix, mirroring the prefix handling on the media path). The gateway serves the bare id, so leaving the prefix on returns 'No available providers for model infron/...'. Only a copy of the Request is rewritten; the caller's Request is untouched. One real difference is handled rather than papered over: Infron returns the native cost at the top level of the body and of the final stream chunk, rather than nested under usage. Parsed naively that leaves Usage.Cost nil, which the cost tracker reads as 'price unknown' -- usage still recorded, but with no cost and an empty cost_source instead of 'provider'. Response/StreamChunk now carry the top-level field and normalizeNativeCost folds it into Usage.Cost, so every existing consumer keeps reading one place. An explicit usage.cost always wins. A gateway key that was already honored before Infron existed keeps precedence, so adding an Infron key never reroutes an existing deployment. llm.onerouter.pro is deliberately NOT added to vouchedRewriteDomains: max_tokens and max_completion_tokens behaved identically in probing and neither could be shown to be enforced, so the conservative legacy max_tokens path stays, per the reasoning already in that comment.
8574fd0 to
6323eaa
Compare
|
Thanks for putting this together. I did an initial pass on the diff, but I can’t take it to merge yet because the required repo checks never showed up on this PR. At the moment I only see , so branch protection is still blocking on missing checks like . Please rebase on current or otherwise retrigger the normal PR workflows, and I’ll do a final mergeability pass once the required checks are actually running. |
|
Thanks for putting this together. I did an initial pass on the diff, but I can’t take it to merge yet because the required repo checks never showed up on this PR. At the moment I only see |
Performance
✓ No regressions detected |
📊 Coverage gateThresholds from
✅ Gate passedNo surface regressed past the allowed threshold and the aggregate stayed above the floor. |
📐 Patch coverage gateThreshold: 80% on lines this PR touches vs
✅ Patch gate passedEvery surface whose lines were touched by this PR has patch coverage at or above the threshold. |
DefaultConfig applied the Infron block unconditionally, so an environment with OPENAI_API_KEY set and INFRON_API_KEY added resolved to the Infron key and base URL. That contradicts the guarantee stated in DefaultConfig's own doc comment and in ENVIRONMENT_VARIABLES.md, and it matters because spawned agent processes inherit the parent environment -- one exported INFRON_API_KEY would move every Go agent's traffic and credential to a different gateway. The existing precedence test cleared OPENAI_API_KEY on its first line, so it only exercised the OpenRouter branch and the gap passed CI green. Adds the regression test for the OpenAI case plus one pinning that Infron still applies when it is the only gateway key set, and names the OpenRouter attribution fallback vars in the docs so operators can audit what feeds the gateway.
AbirAbbas
left a comment
AbirAbbas
left a comment
There was a problem hiding this comment.
Merging this. Thanks for the disclosure up front, for following the existing attribution/config helpers instead of inventing a second shape, and for the live end-to-end runs — that made this much easier to check. Sorry it sat: the required checks never appeared because the workflow runs were stuck behind our first-time-contributor approval gate, and the advice you got to rebase wouldn't have fixed it. That was our mistake.
I pushed one commit before merging. DefaultConfig() applied the Infron block unconditionally, so an environment with OPENAI_API_KEY already set plus INFRON_API_KEY resolved to the Infron key and llm.onerouter.pro — which contradicts the guarantee in your own doc comment and in ENVIRONMENT_VARIABLES.md that adding an Infron key never reroutes an existing deployment. I proved it with a throwaway test before changing anything: APIKey="infron-key" BaseURL="https://llm.onerouter.pro/v1".
Your TestDefaultConfigExistingGatewayWinsOverInfron sets OPENAI_API_KEY to empty on its first line, so it only exercised the OpenRouter branch — which was already correct — and the OpenAI case slipped through green. The fix is an apiKey == "" guard on the Infron branch, plus regression tests for both the OpenAI-wins case and the Infron-still-applies-when-alone case.
I read this as an ordering oversight rather than intent — you clearly built the OpenRouter-wins ordering deliberately and OpenAI just needed the same guard. It mattered enough to fix rather than reword because agent processes inherit the parent environment, so a single exported INFRON_API_KEY would have moved every Go agent's traffic and credential. If you actually intended Infron to take precedence over a direct OpenAI key, say so and we'll reword the docs instead.
Two smaller things in the same commit: a comment on defaultInfronBaseURL noting onerouter.pro is Infron's gateway domain, since nothing in the repo connected the two names and grepping either one dead-ended; and ENVIRONMENT_VARIABLES.md now names the OpenRouter fallback vars explicitly rather than saying "the attribution values documented above", so an operator auditing what feeds the gateway can actually grep for them.
Three other things came up in review and I want to record that they were checked and cleared, so they don't get re-litigated later. normalizeNativeCost being called outside the gateway switch is fine — it early-returns on Cost == nil and no shipped provider sends top-level cost. The attribution fallback to the OpenRouter vars is fine — it's opt-in behind INFRON_API_KEY, has a kill switch, and carries values the README tells operators to set to their public product name. And IsInfron's strings.Contains is character-for-character what IsOpenRouter has always done; tightening both to host matching is our cleanup, not yours.
… (#884) * fix(sdk/go/ai): never fabricate zero-token usage from a top-level cost normalizeNativeCost synthesized an empty Usage{} when a body carried a top-level cost without a usage block. On the streaming path every consumer accumulates usage last-non-nil-wins, so a cost-only chunk arriving after the real usage chunk replaced genuine token counts with zeros — recorded downstream as input=0/output=0 with cost_source "provider", an authoritative-looking row that has lost its tokens. Fold the cost only into a usage block the provider actually sent. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * fix(sdk/go/ai): don't inherit attribution values past their opt-out Infron attribution fell back to the OpenRouter-scoped site URL and app name but never consulted AGENTFIELD_OPENROUTER_ATTRIBUTION, so values a deployment had explicitly suppressed — often internal hostnames or product names — were sent to a different vendor on the first Infron call. Inherit the values only while OpenRouter attribution is enabled; the Infron defaults apply otherwise. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> --------- Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
4 participants
Summary
Adds the Infron gateway to
sdk/go/ai, in the shape this package already uses to describe a gateway, so selecting it is an env-var change rather than a fork.Infron is an OpenAI-compatible gateway serving the standard
<provider>/<model>ids, so nothing about a model's identity changes when it runs there:infron/moonshotai/kimi-k2.6routes the same model the bare id names. I followed the attribution/config helpers already in the package rather than introducing a second way to describe a provider.Disclosure: I work on Infron. Everything below is checkable from the diff and the commands in the test plan.
sdk/go/ai/infron_attribution.goHTTP-Referer/X-Titlepair, and the attribution env vars already configured in a deployment are honored as fallbacks, so a deployment that already declares itself as "AgentField AI" keeps that identity after switching gateways.sdk/go/ai/config.goIsInfron();DefaultConfig()readsINFRON_API_KEYand points athttps://llm.onerouter.pro/v1.sdk/go/ai/client.gosdk/go/ai/model_params.goinfron/prefix before the request goes out.sdk/go/ai/response.gocostfield +normalizeNativeCost()onResponseandStreamChunk. See below.docs/ENVIRONMENT_VARIABLES.md,sdk/go/ai/README.mdsdk/go/ai/infron_attribution_test.goThe one real difference, handled rather than papered over
This package reads native cost from
usage.cost. Infron reports it at the top level of the body, and of the final stream chunk:{ "model": "deepseek/deepseek-v4-flash", "cost": 0.000002, "usage": { "prompt_tokens": 12, "completion_tokens": 9 } } // no usage.costParsed as-is that leaves
Usage.Cost == nil, whichrecordLLMUsagereads as price unknown: the call is still recorded, but with a nil cost and an emptycost_sourceinstead of"provider". Nothing errors, and it only shows up later as a hole in the cost data.ResponseandStreamChunknow carry the top-level field, andnormalizeNativeCost()folds it intoUsage.Coston parse, so every existing consumer keeps reading one place. An explicitusage.costalways wins; the fold only fills a gap.The model prefix is stripped before the wire
infron/is a routing marker for callers that select a gateway by model string, but the gateway serves the bare id, so leaving the prefix on returnsNo available providers for model infron/moonshotai/kimi-k2.6.stripInfronPrefixremoves it inmarshalRequest, mirroring the prefix handling this package already does on the media path. Only a copy of theRequestis rewritten, so the caller'sRequestandconfig.Modelare untouched andIsInfron()still reports the truth.Worth flagging either way: the chat path does not strip the pre-existing gateway prefix today (only the media path does), so a model string carrying that prefix hits the same wall against its own gateway. I left that alone rather than change existing behavior inside a PR about a new provider, but happy to send it separately if you want the two symmetrical.
Backwards compatibility
A gateway key that was already honored before Infron existed keeps precedence. Adding
INFRON_API_KEYto an existing environment never reroutes it;TestDefaultConfigExistingGatewayWinsOverInfronpins that.IsInfron()also does not match a bare shared model id (moonshotai/kimi-k2.6), only the explicitinfron/prefix or the Infron host, so gateways cannot be confused by model alone.Type of change
Usage.Costinstead of being droppedTest plan
Rebased on
mainat4bc8ce7and re-run today.cd sdk/go && go test ./...—agent,ai,client,did,inputs,typesall okcd sdk/go && go test -race ./ai/...— cleangofmt -lclean on every touched file;go vet ./ai/...cleanai.Config, callsai.NewClient, and exercisesComplete()andStreamComplete(). The attribution headers are asserted by putting a capturing reverse proxy in front of the real gateway, so what is checked is what actually went on the wire:One pre-existing failure, unrelated to this PR:
TestOpenCodeConcurrencyLimit_RealSubprocessinsdk/go/harnessfails identically on a clean checkout ofmainon macOS — same test, same parse error, verified side by side before opening this. It shells out todate +%s%N, which BSDdatedoes not support, so the test parses a literalN. It passes in CI on Linux. Happy to send that as a separate fix if useful.Test coverage
sdk-go).coverage-baseline.jsonchange needed, and here is why:mainsdk/go/aistatement coverageMeasured back to back with
-count=1; the number moves about 0.1 pp between runs on its own. That is against amax_surface_dropof 1.0 and amin_surfaceof 84.0, on the smaller of the two numbers feeding thesdk-gosurface. Patch coverage on the non-test lines this PR adds is 92.7% (101/109 coverable added lines), againstmin_patch = 80.0; the new file itself is at 95.7%,IsInfronand the prefix strip at 100%.Notes
llm.onerouter.prois deliberately not added tovouchedRewriteDomains. I probed bothmax_tokensandmax_completion_tokensand they behaved identically, with neither demonstrably enforced, so I left the conservative legacymax_tokenspath in place per the reasoning already in that comment. Easy to add if you have better information.open_codeprovider + theINFRON_API_KEYauto-select path): feat: add Infron as an open_code gateway provider SWE-AF#126. The two are independent; either can land alone.